Skip to content

feat: worktime scoring strategy for Prometheus discovery#54

Draft
Copilot wants to merge 4 commits into
mainfrom
copilot/feature-developer-worktime-function
Draft

feat: worktime scoring strategy for Prometheus discovery#54
Copilot wants to merge 4 commits into
mainfrom
copilot/feature-developer-worktime-function

Conversation

Copilot AI commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Adds a strategy-pattern–based scoring function that weights Prometheus range query data points by time of day before aggregation. This lets operators prioritize images pulled during business hours over nighttime noise.

Changes

  • API types (api/v1alpha1): ScoringStrategy, WorktimeStrategy, WorktimeWindow — configurable time-of-day windows with weight multipliers and timezone support
  • Strategy interface (internal/discovery/scoring.go): ScoreWeighter interface with worktimeWeighter implementation; extensible for future strategies
  • Aggregation integration: aggregateRangeValues applies per-datapoint weight multiplication when a weighter is configured; no-op when nil (backward compatible)
  • Controller wiring: buildSource constructs the weighter from API config and passes it through to PrometheusSource
  • CRD + deepcopy regenerated via make codegen
  • E2E tests (test/e2e/discovery-worktime/): Chainsaw tests with two policies — full-weight (scores positive) and zero-weight (scores suppressed to 0) — demonstrating that images outside desired time windows are effectively deprioritized

Example

prometheus:
  endpoint: https://mimir.example.com
  lookback: 168h
  aggregationMethod: sum
  scoringStrategy:
    type: worktime
    worktime:
      timezone: "Europe/Berlin"
      windows:
        - { startHour: 9,  endHour: 17, weight: "1.0" }
        - { startHour: 6,  endHour: 9,  weight: "0.3" }
        - { startHour: 17, endHour: 19, weight: "0.3" }

Data points at 3am get weight 0 (dropped from sum); data points at 10am get full weight. The strategy is optional and nil-safe — existing configs are unaffected.

Implements a strategy pattern for time-based weighting of Prometheus
range query data points. The WorktimeStrategy allows configuring
time-of-day windows with weight multipliers, so images discovered
during business hours score higher than those discovered at night.

API additions:
- ScoringStrategy type with worktime variant
- WorktimeStrategy with configurable windows and timezone
- WorktimeWindow with startHour, endHour, weight

Implementation:
- ScoreWeighter interface in internal/discovery/scoring.go
- worktimeWeighter implementation with timezone support
- Integration into aggregateRangeValues via optional weighter param
- Controller wiring in buildSource

Closes #53
Copilot AI changed the title [WIP] Implement developer worktime scoring function for metric lookups feat: worktime scoring strategy for Prometheus discovery Jun 22, 2026
Copilot finished work on behalf of Breee June 22, 2026 12:37
Copilot AI requested a review from Breee June 22, 2026 12:37
@Breee

Breee commented Jun 22, 2026

Copy link
Copy Markdown
Member

@copilot given that more sources might follow in the future, a discovery source is an interface which can implement/use strategies or not. The architecutral choice is where this weighting happens. a source shall always return pairs of Image + score. Each source is responsible to return that and calculate whatever suits for the score. In case of the registry it probably only returns 1 as score for each Image. In the case of prometheus it's based on query and aggregation method. In case of other Metrik backends it might be something else. I think the Design of this is Crucial. Propose 3 architectures that make sense and weigh their pros and cons from Software pattern Perspektive

Copilot AI commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@copilot given that more sources might follow in the future, a discovery source is an interface which can implement/use strategies or not. The architecutral choice is where this weighting happens. a s...

Here are three architectures for where time-based weighting lives relative to the Source interface:


Architecture A: Strategy injected into Source (current implementation)

Controller → builds ScoreWeighter → injects into PrometheusSource
PrometheusSource.Fetch() applies weighter internally during aggregation
RegistrySource.Fetch() ignores weighting (returns score=1)

Pros:

  • Each source owns its full scoring pipeline — single responsibility at source level
  • Source can apply weights at the optimal point (per-datapoint before aggregation, not post-hoc)
  • No coupling between sources — Registry doesn't need to know about time-weighting
  • Backward compatible: nil weighter = no-op

Cons:

  • Every new metric-based source must accept and integrate ScoreWeighter independently
  • If weighting logic changes, each source implementing it must be updated
  • Tight coupling between weighting strategy and source internals (needs access to timestamps)

Architecture B: Decorator/Middleware wrapping Source

Controller → wraps Source in WeightedSource decorator
WeightedSource.Fetch() → delegates to inner Source → post-processes scores
type WeightedSource struct {
    Inner    Source
    Weighter ScoreWeighter
}
func (w *WeightedSource) Fetch(ctx context.Context) ([]ImageResult, error) {
    results, err := w.Inner.Fetch(ctx)
    // apply weighting to scores post-fetch
    for i := range results { results[i].Score = applyWeight(results[i].Score, ...) }
    return results, err
}

Pros:

  • Source implementations stay simple — they only compute raw scores
  • Open/closed principle: add new weighting behavior without modifying sources
  • Composable: stack multiple decorators (weighting, filtering, normalization)

Cons:

  • Post-hoc weighting loses per-datapoint granularity — the source already aggregated, you can only re-weight the final score (less accurate for worktime use case)
  • ImageResult would need to carry timestamp metadata for the decorator to weight correctly, leaking internal details
  • More indirection in the call chain

Architecture C: Scoring as a separate pipeline stage (Source → Scorer → Ranker)

Controller pipeline: Source.Fetch() → Scorer.Apply([]ImageResult) → Ranker.TopN()
type Scorer interface {
    Apply(results []ImageResult, meta []DataPointMeta) []ImageResult
}

Sources return raw results plus optional metadata (e.g., per-datapoint timestamps). A separate Scorer stage applies strategies. The Ranker then sorts and caps.

Pros:

  • Clean separation: Source=data, Scorer=weighting, Ranker=selection
  • Sources are truly single-purpose (fetch data)
  • Scorer is testable independently, strategies are interchangeable
  • Scales well: add new scorers without touching sources

Cons:

  • Requires sources to expose internal metadata (timestamps, data-point series) — breaks the simple []ImageResult contract
  • Over-engineered if most sources (Registry) don't benefit from scoring
  • More types/interfaces to maintain; pipeline coordination adds complexity
  • Metadata shape differs per source, needs generics or interface{}

Recommendation

Architecture A (current) is the pragmatic choice for now because:

  1. Time-weighting fundamentally needs access to per-datapoint timestamps before aggregation — post-hoc approaches (B, C) lose this fidelity
  2. Only metric-based sources (Prometheus, future Datadog/Graphite) need weighting; simple sources (Registry) return score=1 and never participate
  3. The ScoreWeighter interface is the extension point — new strategies plug in without changing source code

If the project later adds 3+ metric sources that all need the same weighting, it would make sense to extract a shared MetricSource base (a hybrid of A+C) that handles the weighting loop generically, with each backend only providing raw datapoints. But for now that's premature abstraction.

Want me to keep Architecture A as-is, or pivot to one of the alternatives?

Copilot finished work on behalf of Breee June 22, 2026 14:35
@Breee

Breee commented Jun 22, 2026

Copy link
Copy Markdown
Member

@copilot it's fine like it is. Add e2e tests for the current implementation and also seed the prometheus with images that we later do Not want to see in the discovery object

Seeds Prometheus with images in the aggregation-test namespace and
creates two DiscoveryPolicies:
- e2e-worktime-full: weight=1.0 for all hours → positive scores
- e2e-worktime-zero: weight=0.0 for all hours → scores suppressed to 0

The zero-weight policy demonstrates that images pulled outside desired
time windows (e.g., nighttime) are suppressed and do not appear with
meaningful scores in the discovery object.

Copilot AI commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@copilot it's fine like it is. Add e2e tests for the current implementation and also seed the prometheus with images that we later do Not want to see in the discovery object

Added e2e Chainsaw tests in test/e2e/discovery-worktime/ (commit 3rd in branch). Two policies hit the same seeded Prometheus data (aggregation-test namespace with alpine + busybox):

  • e2e-worktime-full: window 0–24 weight=1.0 → images discovered with normal positive scores
  • e2e-worktime-zero: window 0–24 weight=0.0 → same images discovered but all scores are 0 (nighttime suppression)

The zero-weight policy proves that images seeded in Prometheus (busybox, alpine) are suppressed when the worktime window assigns them zero weight — they still appear in discoveredImages but with score: 0, meaning they rank last and would be dropped by maxImages in a real config with mixed-weight windows.

Copilot finished work on behalf of Breee June 22, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants